Cloud computing platform that executes third-party code in a distributed cloud computing network

ABSTRACT

A compute server receives a first request from a client device that triggers execution of a first third-party code piece. The first request is directed to a first zone. A single process at the compute server executes the first third-party code piece. As a result of executing the first third-party code piece, a second request is generated that triggers execution of a second third-party code piece. The second request is directed to a second zone. The single process executes the second third-party code piece. A response is generated to the first request based at least in part on the executed first third-party code piece and the executed second third-party code piece. The generated response is transmitted to the client device.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of application Ser. No. 17/114,382,filed Dec. 7, 2020, now U.S. Pat. No. 11,561,805, which is acontinuation of application Ser. No. 16/450,585, filed Jun. 24, 2019,now U.S. Pat. No. 10,860,340, which is a continuation of applicationSer. No. 16/182,522, filed Nov. 6, 2018, now U.S. Pat. No. 10,331,462,which are hereby incorporated by reference.

FIELD

Embodiments of the invention relate to the field of network computing;and more specifically, to a cloud computing platform that executesthird-party code in a distributed cloud computing network.

BACKGROUND

Historically, web application code has been split between origin serversand browsers that are connected by a network that transmits data frompoint to point. Many large websites were first run on large physicalmainframe servers that could handle large traffic and large data. Overtime a switch was made to run websites on tens to hundreds of commodityservers that allowed for a reduction in cost, more fault tolerance, andincreased performance. The next switch was using virtual machines whereone physical machine could be split into multiple virtual machines thatcan be independently managed. However, virtual machines typically have ahigh cost. For instance, each virtual machine is typically allocatedhundreds of megabytes of RAM and typically takes tens of seconds toboot. Containers can be used to further provide isolation and are lessresource intensive than virtual machines. But, web application coderunning in a container typically is run in its own OS-level process,consuming RAM and inducing context-switching overhead. Also, whilenative code can load quickly in a container, many server-orientedlanguage environments are not optimized for startup time.

Some cloud computing platform process spin up a containerized processfor your code and auto-scales the process which creates cold-starts. Acold-start occurs when a new copy of the code starts on a machine. A newcontainerized process is begun which can take between hundreds ofmilliseconds to multiple seconds (e.g., between 500 ms to 10 seconds).This means that any request may be hanging for as much time as it takesto begin the new containerized process (e.g., as much as ten seconds).Also, this containerized process can only process a single request at atime and a new containerized process must be cold-started each time anadditional concurrent request is received. This means that a laggyrequest can happen over and over. Also, if the containerized processdoes not receive a request to process within a certain amount of time,it will automatically shut down and will need to be cold-started againonce the request is received. When new code is deployed, this entireprocess proceeds again as each containerized process needs to be spun upanew.

One of the key features of an operating system is the ability to runmany processes at once. The operating system transparently switchesbetween the various processes that want to run code at any given time.The operating system accomplishes this through a context switch thatmoves the memory required for one process out and the memory requiredfor the next process in. A context switch can take as much as 100microseconds. When multiplied by all the processes running on theaverage cloud computing platform server creates a heavy overhead. Thismeans that not all the CPU's power can be devoted to actually executingthe customer code, but rather some is spent switching between theprocesses.

Most computing platforms are meant to be run by individual customers ontheir own servers. They are not intended to be run in a multi-tenantenvironment, executing code of other customers. Memory is often thehighest cost of running a customer's code (even higher than the CPUcost).

Building and maintaining applications that easily scale to supportspikes in demand or a global user base has generally required a largeamount of both upfront engineering work and ongoing operational support.Developers are forced to spend significant time on writing supportingcode rather than building the application itself. Many cloud computingplatforms require the developer to specify where the code should run(e.g., at which nodes of the cloud computing platform), often with asmall number of nodes that can be selected.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention may best be understood by referring to the followingdescription and accompanying drawings that are used to illustrateembodiments of the invention. In the drawings:

FIG. 1 illustrates an exemplary embodiment of a cloud computing platformthat executes third-party code in a distributed cloud computing networkaccording to an embodiment.

FIG. 2 illustrates an example of a compute server according to anembodiment.

FIG. 3 is a flow diagram that illustrates exemplary operations forexecuting third-party code in a distributed cloud computing networkaccording to an embodiment.

FIG. 4 is a flow diagram that illustrates exemplary operations forexecuting third-party code in a distributed cloud computing networkaccording to an embodiment.

FIG. 5 is a conceptual figure that shows a conceptual relationshipbetween third-party code and the process overhead of the isolatedexecution environment model that is described in embodiments.

FIG. 6 is a conceptual figure that shows a conceptual relationshipbetween code and the process overhead of a virtual machine model.

FIG. 7 illustrates a block diagram for an exemplary data processingsystem that may be used in some embodiments.

DESCRIPTION OF EMBODIMENTS

A method and apparatus for a cloud computing platform that executesthird-party code in a distributed cloud computing network is described.The distributed cloud computing network includes multiple computeservers that are geographically distributed (e.g., in differentlocations throughout the world). There may be hundreds of computeservers that are geographically distributed in differentpoints-of-presences (PoPs). Each PoP may include one or more physicalservers (e.g., one or more compute servers, one or more control servers,one or more DNS servers (e.g., one or more authoritative name servers,one or more proxy DNS servers), and one or more other pieces of networkequipment such as router(s), switch(es), and/or hub(s)). Each PoP may bepart of a different data center and/or colocation site. The distributedcloud computing network may provide different services for customers(e.g., domain owners or operators) such as protecting againstinternet-based threats, performance services (e.g., acting as a contentdelivery network (CDN) and dynamically caching customer's files closerto visitors, page acceleration/optimization), TCP stack optimizations,and/or other services.

Third-party code (e.g., written by or deployed by third-parties such ascustomers) can be deployed to all or some of the compute servers of thedistributed cloud computing network. The third-party code can be, forexample, a piece of JavaScript or other dynamically-typed language, aWebAssembly (WASM) compiled piece of code, or other compiled code. In anembodiment, the third-party code is compliant with the W3C standardServiceWorker API. The third-party code can, for example, intercept andanswer HTTP requests and make outgoing HTTP subrequests as part ofanswering an HTTP request. For purposes of this description, each pieceof third-party code is referred to as a worker script and an executedinstance of the worker script is referred to as a worker.

A worker script of a particular third-party is run in an executionenvironment in which a single process can safely run third-party code.The process can contain multiple execution environments at the same timeand the process can seamlessly switch between them. Code in oneexecution environment cannot interfere with code running in a differentexecution environment despite being in the same process. The executionenvironments are managed in user-space rather than by an operatingsystem. Each execution environment uses its own mechanism to ensure safememory access, such as preventing the code from requesting access toarbitrary memory (restricting its use to the objects it has been given)and/or interpreting pointers within a private address space that is asubset of an overall address space. This execution environment is not acontainer or virtual machine. For purposes of description, this type ofexecution environment is sometimes referred herein as an isolatedexecution environment. In a specific implementation, the worker scriptis run in an isolate of the V8 JavaScript engine.

Because a single process can run multiple isolated executionenvironments, the overhead of running the isolated executionenvironments is occurred once (e.g., starting the single process to runthe isolated execution environments) and isolated execution environmentscan be started and run with very little individual overhead. The workerscripts are not executed using a virtual machine or a container. Unlikeother cloud computing platforms that spin up a containerized process forprocessing code that can take as much as ten seconds, an isolatedexecution environment can be started in as little as 5 ms because a newprocess does not need to be started (assuming the single process thatruns the isolated execution environments is started). Also, since theworker scripts can be run in a single process, there are no expensivecontext switches like experienced with other cloud computing platformswhich means that more time is spent running the code instead ofperforming context switches. Also, because the implementations of thesingle process are shared between all isolated execution environments,the memory requirements are less than traditional cloud computingplatforms.

In an embodiment, a particular worker script is loaded and executedon-demand (when and only if it is needed) at a particular compute serverof the distributed cloud computing network. Each request for a domainthat triggers handling of a worker script will be handed by the workerat the compute server that is closest to the requesting user. Forinstance, when responding to a user in New Zealand, a worker script isrun in a compute server in New Zealand that is closest to the user. Thecompute server that receives the request may be determined by thenetwork infrastructure according to an Anycast implementation or by ageographical load balancer.

FIG. 1 illustrates an exemplary embodiment of a cloud computing platformthat executes third-party code in a distributed cloud computing networkaccording to an embodiment. The cloud computing platform providesdifferent services such as protecting against internet-based threats,providing performance services for customers (e.g., acting as a contentdelivery network (CDN) and dynamically caching customer's files close tovisitors, page acceleration, etc.), TCP stack optimizations, and/orother services. The system 100 includes the client devices 110A-N, thecompute servers 120A-N, the control server 130, the origin server 140,and the customer device 150.

Each client device is a computing device (e.g., laptop, workstation,smartphone, mobile phone, tablet, gaming system, set top box, wearabledevice, Internet of Things (IoT) device, etc.) that is capable oftransmitting and/or receiving network traffic. Each client device mayexecute a client network application such as a web browser, nativeapplication, or other application that can access network resources(e.g., web pages, images, word processing documents, PDF files, moviefiles, music files, or other computer files).

The compute servers 120A-N are part of the cloud computing platform. Thecompute servers 120A-N are physical servers and are geographicallydistributed (e.g., in different locations throughout the world). Thecompute servers 120A-N are part of the distributed cloud computingnetwork 105. There may be hundreds of compute servers as part of thecloud computing platform. Although not illustrated in FIG. 1 , thecompute servers 120A-N may be part of PoPs that may include otherphysical servers (e.g., one or more compute servers, one or more controlservers, one or more DNS servers (e.g., one or more authoritative nameservers, one or more proxy DNS servers), and one or more other pieces ofnetwork equipment such as router(s), switch(es), and/or hub(s)). EachPoP (and each compute server) may be part of a different data centerand/or colocation site. Although not illustrated in FIG. 1 , there areother physical devices between the compute servers 120A-N such asrouters, switches, etc.

The control server 130 is operated by the cloud computing platform andprovides a set of tools and interfaces for a customer to, among otherthings, configure code to run in the cloud computing platform. Forinstance, the control server 130 may allow the customer to upload one ormore worker scripts and may allow the customer to specify when theworker script(s) are to be run. For instance, the customer may associatea rule that indicates when a worker script is to be run. By way ofexample, the control server 130 may allow the customer to configure aURL matching pattern that indicates the URL(s) for which the workerscript is to run. The control server 130 may allow the customer todelete and update previously uploaded worker script(s).

The control server 130 deploys each worker script to each of the computeservers 120A-N automatically (without the customer selecting which ofthe compute servers 120A-N in which to deploy the worker script). Inanother embodiment, the control server 130 allows the customer toindicate which of the compute servers 120A-N are to be deployed aparticular worker script. The control server 130 creates an identifierfor each unique worker script. In an embodiment, the identifier iscreated by hashing the content of the worker script (e.g., using acryptographic hash function such as SHA-256), where two scripts withidentical content will have the same identifier even if uploaded bydifferent customers and even if applied to different zones.

In an embodiment, the control server 130 allows a customer to provisionthe service to the cloud computing platform through DNS. For example,DNS record(s) of a customer are changed such that DNS records ofhostnames point to an IP address of a compute server instead of theorigin server. In some embodiments, the authoritative name server of thecustomer's domain is changed to an authoritative name server of theservice and/or individual DNS records are changed to point to thecompute server (or point to other domain(s) that point to a computeserver of the service). For example, the customers may change their DNSrecords to point to a CNAME record that points to a compute server ofthe service. In one embodiment, customers may use the control server 130to change their authoritative name server to an authoritative nameserver of the cloud computing platform and/or change their zone filerecords to have their domain point to the compute servers.

The third-party device 150 is a computing device (e.g., laptop,workstation, smartphone, mobile phone, tablet, etc.) that is used bythird-parties to, among other things, configure their third-party codeto run in the cloud computing platform. A third-party may be a customerof the cloud computing platform and/or a party that is configuringthird-party code to run in the cloud computing platform.

The origin server 140, which may be owned or operated directly orindirectly by the customer of the cloud computing platform, is acomputing device on which a network resource resides and/or originates(e.g., web pages, images, word processing documents, PDF files moviefiles, music files, or other computer files). In an embodiment, theorigin server 140 is not required to be in the cloud computing platform(e.g., third-party code may run on the compute servers withoutcommunicating with an origin server). Although FIG. 1 illustrates theorigin server 140 communicating with the compute server 120A, the originserver 140 may also communicate with one or more of the other computeservers 120B-N.

The compute servers 120A-N are geographically distributed whichdecreases the distance between requesting client devices and the computeservers and decreases the time necessary to respond to a request. Thecompute servers 120A-N may operate as a reverse proxy and receiverequest for network resources (e.g., HTTP requests) of a domain of theorigin server 140. The particular compute server 120 that receives arequest from a client device may be determined by the networkinfrastructure according to an Anycast implementation or by ageographical load balancer. For instance, the compute servers 120A-N mayhave a same anycast IP address for a domain of the origin server 140. Ifthe origin server 140 handles the domain “example.com”, a DNS requestfor “example.com” returns an address record having the anycast IPaddress of the compute servers 120A-N. Which one of the compute servers120A-N receives a request from a client device depends on which computeserver 120 is closest to the client device in terms of routing protocolconfiguration (e.g., Border Gateway Protocol (BGP) configuration)according to an anycast implementation as determined by the networkinfrastructure (e.g., router(s), switch(es), and/or other networkequipment between the requesting client and the compute servers 120A-N.By way of example, the client device 110A is closest to the computeserver 120A, the client device 110B is closest to the compute server120B, and the client device 110L is closest to the compute server 120N.Accordingly, requests from the client device 110A are received at thecompute server 120A, requests from the client device 110B are receivedat the compute server 120B, and requests from the client device 110L arereceived at the compute server 120N. In some embodiments, instead ofusing an anycast mechanism, a geographical load balancer is used toroute traffic to the nearest compute server. The number of clientdevices and compute servers illustrated in FIG. 1 is exemplary. Thedistributed cloud computing network 105 may include hundreds tothousands (or more) compute servers and each compute server may receiverequests from thousands or more client devices.

In the example of FIG. 1 , each of the compute servers 120A-N canexecute the worker script(s) of a third-party. Each worker script is runin an isolated execution environment, such as run in an isolate of theV8 JavaScript engine. Thus, as illustrated in FIG. 1 , the computeserver 120A includes the isolated execution environments 130A-N thateach executes a separate worker script 135. The isolated executionenvironment 130A-N are run within a single process. The worker scriptsare not executed using a virtual machine or a container. In anembodiment, a particular worker script is loaded and executed on-demand(when and only if it is needed) at a particular compute server of thedistributed cloud computing network. Each request for a domain thattriggers handling of a worker script will be handed by the worker at thecompute server that is closest to the requesting user.

FIG. 2 illustrates an example of a compute server 120 according to anembodiment. The compute server 120A includes a gateway module 210, aworker process 215, a cache 220, and a data store 240. The gatewaymodule 210 receives web requests and processes web responses (e.g., HTTPrequests and HTTP responses). The gateway module 210 may be acting as areverse proxy for the origin server 140. The gateway module 210 hasaccess to the cache 220 that is configured to cache network resources(e.g., web pages, images, word processing documents, PDF files moviefiles, music files, or other computer files), configuration files,scripts, and/or other computer files. The data store 240 is configuredto store, among other items, the worker scripts that are received fromthe control server 130, the URL matching pattern rule that indicates theURL(s) for which the worker script is to run, and a worker scriptmapping that maps worker script identifiers to URL matching patterns.

The worker process 215 is a single process that executes the isolatedexecution environments 130A-N. There may be hundreds to thousands ofisolated execution environments that are run simultaneously by theworker process 215. Each different worker script 135 is run by adifferent one of the isolated execution environments 130A-N each withits own heap. The worker process 215 starts an isolated executionenvironment to load a particular worker script on the first use of theworker script. Depending on the complexity of the worker script, loadingthe worker script may take approximately tens to hundreds ofmilliseconds of CPU time. A worker script says loaded in memory betweenrequests so that the worker script can be used to respond quickly when anew request that triggers that worker script arrives. Handling a singlerequest for a worker script that is already loaded typically takes afraction of a millisecond of CPU time depending on the complexity of theworker script. In an embodiment, one isolated execution environment iscreated per unique worker script identifier. Thus, if many zones useidentical worker scripts, memory is saved by compiling the script onlyonce. The worker process 215 evicts worker scripts (e.g., in a leastrecently used fashion). The worker process 215 may be started duringbooting of the compute server 120A or when the first worker script istriggered for execution.

The gateway module 210 receives a request from the client device 110A.The request may be an HTTP request for a zone of the customer. Thegateway module 210 processes the request including determining whetherthe request triggers executing of a worker script. For instance, thegateway module 210 analyzes the request URL against the URL matchingpattern configured for the zone to determine if a worker script is to beexecuted. If a worker script is to be executed, the gateway module 210annotates the request with the identifier of the worker script to beexecuted as determined by the script mapping table and forwards therequest to the worker process 215. If the identified worker script isalready loaded (if there is already an isolated execution environmentrunning an instance of the worker script), the worker process 215 doesnot need to load another instance of the worker script. However, if theidentified worker script that is already loaded is from a different zone(which is probably from a different customer), the worker process 215creates a separate context (a global object) for the worker script sothat each zone has its own isolated global state. That prevents zonesfrom interfering with the state of other zones. The gateway module 210generates the response after the worker script(s) are executed. If theidentified worker script is not loaded, the worker process 215 createsan isolated execution environment and loads and executes the workerscript.

The executed worker script can take various actions depending on how thescript is written. The worker script may make one or more furtherrequests (referred herein as “subrequests”) such as additional HTTPrequests. These subrequests may be destined for the origin server 140 orto other destinations on the internet. The worker process 215 sends thesubrequests back to the gateway module 210 for further processing. Thegateway module 210 is configured to prevent the subrequest from loopingback to the same worker script. But, the subrequest may trigger adifferent worker script potentially from a different zone. If thesubrequest is to the same zone, the gateway module 210 transmits thesubrequest to the origin server 140 for processing and receives theresponse from the origin server 140. If the subrequest triggers a workerscript, the gateway module 210 annotates the request with the identifierof the worker script to be executed as determined by the script mappingtable and forwards the request to the worker process 215 for executingthe script.

Thus, a single request can trigger multiple worker scripts, even fromdifferent zones from different customers, to be run. Since the workerscripts are run on the same physical compute server, network latency toexecute these scripts reduces to zero. Further, there is savings inbandwidth because the different origin zones may not need to becontacted. To provide an example, say a customer has a service where auser can make a purchase by an SMS message. The user sends an SMSmessage to make the purchase where the SMS message is handled by an APIof a first third-party provider that generates an event to a function.That function invokes a payment API of a second third-party provider tocharge the user's credit card, which generates an invoice event that ishandled by a function to email the invoice to the user. Each of thesetransactions involve potentially crossing the internet, incurringlatency and bandwidth charges. In contrast, with embodiments describedherein, the API of the first third-party provider and the API of thesecond third-party provider can be implemented as worker scripts andexecuted on the same physical compute server, reducing latency andbandwidth.

The worker scripts can perform many different actions. By way ofexample, the worker scripts may perform one or more of the following:intercept and modify HTTP request and response URLs, status, headers,and body content; respond to requests directly from the worker script orforward the request elsewhere; send HTTP requests to third-partyservers; send multiple requests, in serial or in parallel, and use theresponses to compose a final response to the original request; sendasynchronous requests after the response has already been returned tothe client (for example, for logging or analytics); and control behaviorsuch as caching behavior. A customer may perform one or more of theseactions to do the following, for example: perform A/B testing betweentwo different back-ends; build “serverless” applications that relyentirely on web APIs; create custom security filters to block unwantedtraffic; rewrite requests to improve cache hit rate; implement customload balancing and failover logic; and/or collecting analytics withoutrunning code in the user's browser. Of course, these are just examplesand the worker scripts can be used to perform other actions.

FIG. 3 is a flow diagram that illustrates exemplary operations forexecuting third-party code in a distributed cloud computing networkaccording to an embodiment. The operations of FIG. 3 are described withrespect to the exemplary embodiment of FIG. 1 . However, the operationsof FIG. 3 can be performed by different embodiments than those of FIG. 1, and the embodiment described in FIG. 1 can perform operationsdifferent than those of FIG. 3 . The operations of FIG. 3 will bedescribed with respect to HTTP/S request and responses. But, theoperations of FIG. 3 can be performed with different types of requestsand responses.

At operation 305, a first one of multiple compute servers of adistributed cloud computing network receives a request that triggersexecution of a first code piece. The request may be received by a clientdevice and be an HTTP or HTTPS request, for example, destined for azone. The first code piece is one of multiple code pieces that can beexecuted by the first compute server. The first code piece may be athird-party code piece (written and/or provided by an owner or operatorof the zone). The first code piece can be, for example, a piece ofJavaScript or other dynamically-typed language, a WASM compiled piece ofcode, or other compiled code. The first compute server may determinethat the request triggers execution of the first code piece by matchingthe zone to a predetermined matching pattern that associates the firstcode piece to the predetermined matching pattern. For instance, thecompute server may analyze the request URL against a URL matchingpattern configured for the zone to determine if a code piece is to beexecuted and if so, which code piece. With respect to FIG. 1 , thecompute server 120A receives a request from the client device 120A thattriggers execution of a first code piece. To provide an example, theHTTP request is an HTTP GET method for “example.com”, which matches apredefined matching pattern to apply to a code piece with a firstidentifier.

The distributed cloud computing network includes multiple computeservers including the first compute server. The compute servers aregeographically distributed. There may be hundreds or more computeservers. In an embodiment, each of the compute servers are anycasted toa same IP address, and the first compute server receives the request inoperation 305 because the first compute server is the closest one of thecompute servers to the client device making the request as determined byan anycast implementation. For instance, a DNS request for an addressrecord of the zone “example.com” returns an anycasted IP address of thecompute servers. Alternatively, the first compute server may receive therequest as a result of a geographical load balancer routing the requestto it.

In an embodiment, each of the compute servers of the distributed cloudcomputing network includes the first code piece. The first code piececan be deployed to each of the compute servers without the owner oroperator of the zone selecting which of the compute servers are toreceive and potentially run the first code piece. In an alternativeembodiment, the owner or operator of the zone selects which of thecompute servers are to receive and potentially run the first piece.

Next, at operation 310, a single process of the first compute serverexecutes the first code piece in a first isolated execution environment.Multiple other code pieces are being executed by the single process inmultiple other isolated execution environments respectively. Code in oneisolated execution environment cannot interfere with code running in adifferent execution environment despite being in the same process. Thecode pieces may be triggered to execute for different zones anddifferent customers. The isolated execution environment is managed inuser-space rather than by an operating system. Data cannot be shared ormoved across isolated execution environments (each isolated executionenvironment has a completely separate state). The single process on thecompute server can run multiple isolated execution environments (e.g.,hundreds to thousands) seamlessly switching between them. Each isolatedexecution environment uses its own mechanism to ensure safe memoryaccess, such as preventing the code from requesting access to arbitrarymemory (restricting its use to the objects it has been given) and/orinterpreting pointers within a private address space that is a subset ofan overall address space. In a specific implementation, the first codepiece is run in an isolate of the V8 JavaScript engine.

In an embodiment, the first code piece is loaded and executed on-demand(when and only if it is triggered to execute). That is, first code piecewill not be loaded into memory until and unless a request triggers itfor execution. The first code piece stays loaded in memory (at least fora certain amount of time) between requests so that the first code piececan be used to respond quickly if a new request is received thattriggers execution of that same first code piece. An eviction processmay evict code pieces in a least recently used fashion. If the firstcode piece is not already loaded in memory and/or an isolated executionenvironment is not running for to execute the first code piece, anisolated execution environment is created, and/or the first code pieceis loaded to memory. Depending on the complexity of the first codepiece, loading the first code piece may take approximately tens tohundreds of milliseconds of CPU time.

Executing the first code piece can take many actions depending on theinstructions of the first code piece, including actions to improveperformance, enhance security, and/or increase reliability. As anexample for improving performance, the first code piece can perform thefollowing: use custom logic to decide which if the request is cacheableat the compute server, and canonicalize the requests to improve cachehit rate; expand HTML templates directly on the compute server, fetchingonly dynamic content from the origin server; respond to statelessrequests directly from the compute server without contacting the originserver; and/or split one request into multiple parallel requests todifferent servers, then combine the responses into a single response tothe client. As examples for enhancing security, the first code piece canperform the following: implement custom security rules and filter;and/or implement custom authentication and authorization mechanism. Asexamples for increasing reliability, the first code piece can performthe following: deploy fast fixes to the website without having to updatethe code on the origin server; implement custom load balancing andfailover logic; and respond dynamically when the origin server isunreachable.

The execution of the first code piece may cause a subrequest to begenerated. A subrequest is an additional request such as an additionalHTTP request. The subrequest can be destined for the origin server ofthe zone or can be destined to a different server on the Internet. Ifthe subrequest is for the zone, the first compute server is configuredto prevent the subrequest from looping back to the same first piece ofcode. Instead, the first compute server transmits the subrequest to theorigin server for processing.

If the subrequest is for a different zone that handled by the firstcompute server (e.g., a DNS request for an address record of thedifferent zone returns an anycasted IP address of the compute servers),the subrequest can be processed by the first compute server directly(without the subrequest being transmitted to another compute server, forexample). Processing of this subrequest may trigger a different codepiece to be executed. If the subrequest is for a different zone that isnot handled by the first compute server (e.g., a DNS request for anaddress record of the different zone does not return an IP address ofthe first compute server), the compute server transmits the subrequestto that different server. In an embodiment, a code piece is limited to apredefined number of subrequests. A header stores a number that countsthe number of subrequests and if it exceeds the limit, the subrequestwill not be processed.

Next, at operation 315, the first compute server generates a response tothe request based at least in part on the executed first code piece. Theresponse may be an HTTP response, for example. The content of theresponse depends on the execution of the first code piece. In anembodiment, the execution of the first code piece itself returns an HTTPresponse. The first code piece can be written to respond to the requestdirectly (with no subrequests). Alternatively, the first code piece canbe written to make one or more subrequests and generate the responsewith the results of the subrequest(s). Next, at operation 320, the firstcompute server transmits the generated response to the requesting clientdevice.

FIG. 4 is a flow diagram that illustrates exemplary operations forexecuting third-party code in a distributed cloud computing networkaccording to an embodiment. The operations of FIG. 4 are described withrespect to the exemplary embodiment of FIGS. 1 and 2 . However, theoperations of FIG. 4 can be performed by different embodiments thanthose of FIGS. 1 and 2 , and the embodiment described in FIGS. 1 and 2can perform operations different than those of FIG. 4 . The operationsof FIG. 4 will be described with respect to HTTP/S request andresponses. But, the operations of FIG. 4 can be performed with differenttypes of requests and responses.

At operation 405, a first one of multiple compute servers of adistributed cloud computing network receives a request. The request maybe received by a client device and be an HTTP or HTTPS request, forexample, destined for a zone. In an embodiment, the compute serverreceives the request out of the multiple compute servers because it hasbeen determined to be the closest to the requesting client device asdetermined by an anycast implementation. Instead of the request beingreceived from an external client device, the request could be receivedas a subrequest sent by a third-party code piece. If a subrequest, thesubrequest may identify the third-party code piece that generated thesubrequest (e.g., it may include the identifier of the third-party codepiece).

With respect to FIG. 2 , the gateway module 210 receives the request.Next at operation 310, the gateway module 210 determines whether therequest triggers execution of a third-party code piece. The gatewaymodule 210 may determine that the request triggers execution of athird-party code piece by matching the zone to a predetermined matchingpattern that associates the third-party code piece to the predeterminedmatching pattern. For instance, the gateway module 210 may analyze therequest URL against a URL matching pattern configured for the zone todetermine if a third-party code piece is to be executed and if so, whichthird-party code piece. If the request does not trigger execution of athird-party code piece, then operations move to operation 420. If therequest triggers execution of a third-party code piece, then operationsmove to operation 415.

The gateway module 210 is configured to prevent a subrequest fromlooping back to the same third-party code piece. Thus, at operation 415,the gateway module 210 determines if the request is from a subrequestmade from execution of a third-party code piece and would triggerexecution of the same third-party code piece. For example, the gatewaymodule 210 determines whether the request identifies the third-partycode piece as generating the request and determines whether the requesttriggers execution of the same third-party code piece using thepredetermined matching pattern. If the request is a subrequest and wouldtrigger execution of the same third-party code piece, then flow moves tooperation 420. Otherwise, flow moves to operation 425.

At operation 420, the request is processed without executing athird-party code piece. The request may be processed differentlydepending on the destination of the request and the requester. Forinstance, if the request is for a web page from a client device, thegateway module 210 may access the web page and/or the resource(s) of theweb page from the cache 220 (if available) and/or from the origin serverof the domain of the web page. If the request is from a third-party codepiece (a subrequest), the gateway module 210 processes the request andresponse and returns the data to the third-party code piece for furtherprocessing. After processing the request, flow moves to operation 440for generating the response.

In an embodiment, third-party code pieces stay loaded in memory betweenrequests so that they can be used to respond quickly when a new requestthat triggers the third-party code piece arrives. A third-party codepiece may be run separately per zone. Each different zone running thesame third-party code piece may be put in a separate context (have theirown global object) within the same isolated execution environmentrunning that third-party code piece so each zone as its own isolatedglobal state. This prevents two different zones from interfering withthe states of each other while allowing the contexts to share resources.At operation 425, the gateway module 210 determines whether thethird-party code piece is loaded into memory. If the third-party codepiece is loaded (e.g., if there is already an isolated executionenvironment running for the third-party code piece for the zone), thethird-party code piece is executed in operation 435. If the third-partycode piece is not loaded, then at operation 430 the third-party codepiece is loaded and then the third-party code piece is executed inoperation 435.

The executed third-party code piece can take various actions dependingon how the code is written. The third-party code piece may make one ormore subrequests that can be destined for an origin server of the zoneof the third-party code piece or to other destinations on the internet.As illustrated in FIG. 4 , if executing the code piece 435 causes asubrequest to be generated, the subrequest is received at the gatewaymodule 210 in operation 405. The subrequest identifies the third-partycode piece that generated the subrequest. The result of the subrequestmay be processed by the code piece.

At operation 440, a response is generated. The response may be an HTTPresponse, for example. If a third-party code piece was executed, thecontent of the response depends on the execution of that third-partycode piece. The response is sent to the requesting client.

Running third-party code pieces in isolated execution environmentsimproves the performance as compared with running code using a virtualmachine or a container. Unlike other computing platforms that spin up acontainerized process for processing code that can take as much as tenseconds, an isolated execution environment can be started in as littleas 5 ms because a new process does not need to be started. Thus, theoverhead of running an isolated execution environment is small ascompared to other computing platforms. Further, since the third-partycode can be run in a single process, there are no expensive contextswitches like experienced with other computing platforms which meansthat more time is spent actually running the code instead of performingcontext switches. FIG. 5 is a conceptual figure that shows a conceptualrelationship between third-party code and the process overhead of theisolated execution environment model that is described in embodimentsherein. FIG. 6 is a conceptual figure that shows a conceptualrelationship between code and the process overhead of a virtual machinemodel. As seen in FIGS. 5 and 6 , the process overhead of the virtualmachine model is experienced for each different code piece (a newprocess has to be started for each different code piece), whereas theprocess overhead of the isolated execution environment model isexperienced once.

FIG. 7 illustrates a block diagram for an exemplary data processingsystem 700 that may be used in some embodiments. Data processing system700 includes one or more processors 705 and connected system components(e.g., multiple connected chips). One or more such data processingsystems 700 may be utilized to implement the embodiments and operationsdescribed with respect to the compute server, control server, or otherelectronic device.

The data processing system 700 is an electronic device which stores andtransmits (internally and/or with other electronic devices over anetwork) code (which is composed of software instructions and which issometimes referred to as computer program code or a computer program)and/or data using machine-readable media (also called computer-readablemedia), such as machine-readable storage media 710 (e.g., magneticdisks, optical disks, read only memory (ROM), flash memory devices,phase change memory) and machine-readable transmission media (alsocalled a carrier) (e.g., electrical, optical, radio, acoustical or otherform of propagated signals—such as carrier waves, infrared signals),which is coupled to the processor(s) 705. For example, the depictedmachine-readable storage media 710 may store program code 730 that, whenexecuted by the processor(s) 705, causes the data processing system 700to execute the gateway module 210 and/or the worker process 215.

The data processing system 700 also includes one or more input or output(“I/O”) devices and interfaces 725, which are provided to allow a userto provide input to, receive output from, and otherwise transfer data toand from the system. These I/O devices 725 may include a mouse, keypad,keyboard, a touch panel or a multi-touch input panel, camera, framegrabber, optical scanner, an audio input/output subsystem (which mayinclude a microphone and/or a speaker), other known I/O devices or acombination of such I/O devices. The I/O devices and interfaces 725 mayinclude wireless transceivers, such as an IEEE 802.11 transceiver, aninfrared transceiver, a Bluetooth transceiver, a wireless cellulartelephony transceiver (e.g., 2G, 3G, 4G, 5G), an NFC transceiver, oranother wireless protocol to connect the data processing system 700 withanother device, external component, or a network and receive storedinstructions, data, tokens, etc. For instance, a wired or wirelesstransceiver may transmit and receive messages to and from the computeserver as described herein.

Additional components, not shown, may also be part of the system 700,and, in certain embodiments, fewer components than that shown in FIG. 7may also be used in a data processing system 700. One or more buses maybe used to interconnect the various components shown in FIG. 7 .

Thus, an electronic device (e.g., a computer or a mobile client device)includes hardware and software, such as a set of one or more processorscoupled to one or more machine-readable storage media to store code forexecution on the set of processors and/or to store data. For instance,an electronic device may include non-volatile memory containing the codesince the non-volatile memory can persist the code even when theelectronic device is turned off, and while the electronic device isturned on that part of the code that is to be executed by theprocessor(s) of that electronic device is copied from the slowernon-volatile memory into volatile memory (e.g., dynamic random accessmemory (DRAM), static random access memory (SRAM)) of that electronicdevice. Typical electronic devices also include a set or one or morephysical network interface(s) to establish network connections (totransmit and/or receive code and/or data using propagating signals) withother electronic devices. One or more parts of an embodiment of theinvention may be implemented using different combinations of software,firmware, and/or hardware.

The techniques shown in the figures can be implemented using code anddata stored and executed on one or more computing devices (e.g., clientdevice, compute server, DNS server, control server, origin server,etc.). Such computing devices store and communicate (internally and/orwith other computing devices over a network) code and data usingmachine-readable media, such as non-transitory machine-readable storagemedia (e.g., magnetic disks; optical disks; random access memory; readonly memory; flash memory devices; phase-change memory) andmachine-readable communication media (e.g., electrical, optical,acoustical or other form of propagated signals—such as carrier waves,infrared signals, digital signals, etc.). In addition, such computingdevices typically include a set of one or more processors coupled to oneor more other components, such as one or more storage devices, userinput/output devices (e.g., a keyboard, a touchscreen, and/or adisplay), and network connections. The coupling of the set of processorsand other components is typically through one or more busses and bridges(also termed as bus controllers). The storage device and signalscarrying the network traffic respectively represent one or moremachine-readable storage media and machine-readable communication media.Thus, the storage device of a given computing device typically storescode and/or data for execution on the set of one or more processors ofthat computing device. Of course, one or more parts of an embodiment ofthe invention may be implemented using different combinations ofsoftware, firmware, and/or hardware.

In the preceding description, numerous specific details are set forth.However, it is understood that embodiments may be practiced withoutthese specific details. In other instances, well-known circuits,structures and techniques have not been shown in detail in order not toobscure the understanding of this description. Those of ordinary skillin the art, with the included descriptions, will be able to implementappropriate functionality without undue experimentation.

References in the specification to “one embodiment,” “an embodiment,”“an example embodiment,” etc., indicate that the embodiment describedmay include a particular feature, structure, or characteristic, butevery embodiment may not necessarily include the particular feature,structure, or characteristic. Moreover, such phrases are not necessarilyreferring to the same embodiment. Further, when a particular feature,structure, or characteristic is described in connection with anembodiment, it is submitted that it is within the knowledge of oneskilled in the art to effect such feature, structure, or characteristicin connection with other embodiments whether or not explicitlydescribed.

While the flow diagrams in the figures show a particular order ofoperations performed by certain embodiments of the invention, it shouldbe understood that such order is exemplary (e.g., alternativeembodiments may perform the operations in a different order, combinecertain operations, overlap certain operations, etc.).

While the invention has been described in terms of several embodiments,those skilled in the art will recognize that the invention is notlimited to the embodiments described, can be practiced with modificationand alteration within the spirit and scope of the appended claims. Thedescription is thus to be regarded as illustrative instead of limiting.

What is claimed is:
 1. A method, comprising: receiving, at a computeserver from a client device, a first request that triggers execution ofa first one of a plurality of third-party code pieces, and wherein thefirst request is directed to a first zone; executing, by a singleprocess at the compute server, the first one of the plurality ofthird-party code piece; generating, as a result of the executing thefirst one of the plurality of third-party code pieces, a second requestthat triggers execution of a second one of the plurality of third-partycode pieces, wherein the second request is directed to a second zone;executing, by the single process, the second one of the plurality ofthird-party code pieces; generating a response to the first requestbased at least in part on the executed first one of the plurality ofthird-party code pieces and the executed second one of the plurality ofthird-party code pieces; and transmitting the generated response to theclient device.
 2. The method of claim 1, wherein executing the first oneof the plurality of third-party code pieces further causes a thirdrequest to be generated and transmitted to an origin server of the firstzone.
 3. The method of claim 1, wherein generating the response isperformed without transmitting a request to an origin server of thefirst zone.
 4. The method of claim 1, further comprising: determiningthe first request triggers execution of the first one of the pluralityof third-party code pieces by matching the first zone to a predeterminedmatching pattern that associates the first one of the plurality ofthird-party code pieces to the predetermined matching pattern.
 5. Themethod of claim 1, wherein the compute server is one of a plurality ofcompute servers that are each anycasted to a same IP address, andwherein the compute server receives the first request because it is aclosest one of the plurality of compute servers to the client device asdetermined by an anycast implementation.
 6. The method of claim 5,wherein each of the plurality of compute servers include the pluralityof third-party code pieces.
 7. The method of claim 1, wherein the firstone of the plurality of third-party code pieces is provided by an owneror operator of the first zone.
 8. A non-transitory machine-readablestorage medium that provides instructions that, when executed by aprocessor, cause the processor to perform operations comprising:receiving, at a compute server from a client device, a first requestthat triggers execution of a first one of a plurality of third-partycode pieces, and wherein the first request is directed to a first zone;executing, by a single process at the compute server, the first one ofthe plurality of third-party code piece; generating, as a result of theexecuting the first one of the plurality of third-party code pieces, asecond request that triggers execution of a second one of the pluralityof third-party code pieces, wherein the second request is directed to asecond zone; executing, by the single process, the second one of theplurality of third-party code pieces; generating a response to the firstrequest based at least in part on the executed first one of theplurality of third-party code pieces and the executed second one of theplurality of third-party code pieces; and transmitting the generatedresponse to the client device.
 9. The non-transitory machine-readablestorage medium of claim 8, wherein executing the first one of theplurality of third-party code pieces further causes a third request tobe generated and transmitted to an origin server of the first zone. 10.The non-transitory machine-readable storage medium of claim 8, whereingenerating the response is performed without transmitting a request toan origin server of the first zone.
 11. The non-transitorymachine-readable storage medium of claim 8, wherein the operationsfurther comprise: determining the first request triggers execution ofthe first one of the plurality of third-party code pieces by matchingthe first zone to a predetermined matching pattern that associates thefirst one of the plurality of third-party code pieces to thepredetermined matching pattern.
 12. The non-transitory machine-readablestorage medium of claim 8, wherein the compute server is one of aplurality of compute servers that are each anycasted to a same IPaddress, and wherein the compute server receives the first requestbecause it is a closest one of the plurality of compute servers to theclient device as determined by an anycast implementation.
 13. Thenon-transitory machine-readable storage medium of claim 12, wherein eachof the plurality of compute servers include the plurality of third-partycode pieces.
 14. The non-transitory machine-readable storage medium ofclaim 8, wherein the first one of the plurality of third-party codepieces is provided by an owner or operator of the first zone.
 15. Acompute server, comprising: a set of one or more processors; and anon-transitory machine-readable storage medium that providesinstructions that, when executed by the set of processors, cause the setof compute server to perform the following operations: receiving, at thecompute server from a client device, a first request that triggersexecution of a first one of a plurality of third-party code pieces, andwherein the first request is directed to a first zone; executing, by asingle process at the compute server, the first one of the plurality ofthird-party code piece; generating, as a result of the executing thefirst one of the plurality of third-party code pieces, a second requestthat triggers execution of a second one of the plurality of third-partycode pieces, wherein the second request is directed to a second zone;executing, by the single process, the second one of the plurality ofthird-party code pieces; generating a response to the first requestbased at least in part on the executed first one of the plurality ofthird-party code pieces and the executed second one of the plurality ofthird-party code pieces; and transmitting the generated response to theclient device.
 16. The compute server of claim 15, wherein executing thefirst one of the plurality of third-party code pieces further causes athird request to be generated and transmitted to an origin server of thefirst zone.
 17. The compute server of claim 15, wherein generating theresponse is performed without transmitting a request to an origin serverof the first zone.
 18. The compute server of claim 15, wherein theoperations further comprise: determining the first request triggersexecution of the first one of the plurality of third-party code piecesby matching the first zone to a predetermined matching pattern thatassociates the first one of the plurality of third-party code pieces tothe predetermined matching pattern.
 19. The compute server of claim 15,wherein the compute server is one of a plurality of compute servers thatare each anycasted to a same IP address, and wherein the compute serverreceives the first request because it is a closest one of the pluralityof compute servers to the client device as determined by an anycastimplementation.
 20. The compute server of claim 19, wherein each of theplurality of compute servers include the plurality of third-party codepieces.
 21. The compute server of claim 15, wherein the first one of theplurality of third-party code pieces is provided by an owner or operatorof the first zone.